Génération automatique de tests unitaires avec Praspel, un langage de spécification pour PHP

The works presented in this memoir are about the validation of PHPprograms through a new specification language, along with its tools. These works follow three axes: specification language, automatic test data generation and automatic unit test generation. The first contribution is Praspel, a new specification language for PHP, based on the Design by Contract. Praspel specifies data with realistic domains, which are new structures allowing to validate and generate data. Based on a contract, we are able to perform Contract-based Testing, i.e.using contracts to automatically generate unit tests. The second contribution isabout test data generation. For booleans, integers and floating point numbers, auniform random generation is used. For arrays, a dedicated constraint solver has been implemented and used. For strings, a grammar description language along with an LL(⋆) compiler compiler and several algorithms for data generation are used. Finally, the object generation is supported. The third contribution defines contract coverage criteria. These latters provide test objectives. All these contributions are implemented and experimented into tools distributed to the PHP community.Les travaux présentés dans ce mémoire portent sur la validation de programmes PHP à travers un nouveau langage de spécification, accompagné de ses outils. Ces travaux s’articulent selon trois axes : langage de spécification, génération automatique de données de test et génération automatique de tests unitaires.La première contribution est Praspel, un nouveau langage de spécification pour PHP, basé sur la programmation par contrat. Praspel spécifie les données avec des domaines réalistes, qui sont des nouvelles structures permettant de valider etgénérer des données. À partir d’un contrat écrit en Praspel, nous pouvons faire du Contract-based Testing, c’est à dire exploiter les contrats pour générer automatiquement des tests unitaires. La deuxième contribution concerne la génération de données de test. Pour les booléens, les entiers et les réels, une génération aléatoire uniforme est employée. Pour les tableaux, un solveur de contraintes a été implémenté et utilisé. Pour les chaînes de caractères, un langage de description de grammaires avec un compilateur de compilateurs LL(⋆) et plusieurs algorithmes de génération de données sont employés. Enfin, la génération d’objets est traitée.La troisième contribution définit des critères de couverture sur les contrats.Ces derniers fournissent des objectifs de test. Toutes ces contributions ont été implémentées et expérimentées dans des outils distribués à la communauté PHP

A Constraint Solver for PHP Arrays

International audienceIn previous works, we have proposed Praspel, a framework for contract-based testing in PHP. Among others, it includes a specification language and a unit test generator which automatically generates test data from formal preconditions. The generator sometimes rejects data, when they do not satisfy parts of the preconditions. In many cases, generation with rejection is not efficient enough. Thus we investigate practical contexts where more efficient generation algorithms can be designed and we extend Praspel with their implementation. After strings, that we have already considered, the most frequent data type in PHP is arrays. They cover most of the needs for collections, because they can store key-value pairs of any kind, they do not have a specific length or depth, and they are efficiently implemented. In this paper, we report on a study to know what are the most popular constraints on PHP arrays. Then we formalize these constraints and we present an implementation in PHP of a constraint solver for these constraints. In this context, the constraint-based approach removes all the rejections

Praspel: Contract-Driven Testing for PHP using Realistic Domains

We present an integrated contract-based testing framework for PHP. It relies on a behavioral interface specification language called Praspel, for "PHP Realistic Annotation and Specification Language". Using Praspel developers can easily annotate their PHP scripts with formal contracts, namely class invariants, and method pre- and postconditions. These contracts describe assertions either by predicates or by assigning realistic domains to data. Realistic domains introduce types in PHP and describe complex structures frequently encountered in applications, such as email addresses or SQL queries. Realistic domains display two properties: predicability, which allows to check if a data belongs to a given realistic domain, and samplability, which allows to generate valid data. This paper introduces coverage criteria dedicated to contracts, designed to exhibit relevant behaviors of the annotated methods. Test data are then computed to satisfy these coverage criteria, by using dedicated data generators for complex realistic domains, such as arrays or strings. This framework has been implemented and disseminated within the PHP community, which gave us feedback on their usage of the tool and the relevance of this integrated process with respect to their practice of manual testing

Praspel: Contract-Driven Testing for PHP using Realistic Domains

We present an integrated contract-based testing framework for PHP. It relies on a behavioral interface specification language called Praspel, for "PHP Realistic Annotation and Specification Language". Using Praspel developers can easily annotate their PHP scripts with formal contracts, namely class invariants, and method pre- and postconditions. These contracts describe assertions either by predicates or by assigning realistic domains to data. Realistic domains introduce types in PHP and describe complex structures frequently encountered in applications, such as email addresses or SQL queries. Realistic domains display two properties: predicability, which allows to check if a data belongs to a given realistic domain, and samplability, which allows to generate valid data. This paper introduces coverage criteria dedicated to contracts, designed to exhibit relevant behaviors of the annotated methods. Test data are then computed to satisfy these coverage criteria, by using dedicated data generators for complex realistic domains, such as arrays or strings. This framework has been implemented and disseminated within the PHP community, which gave us feedback on their usage of the tool and the relevance of this integrated process with respect to their practice of manual testing

The art of contract-based testiong in PHP with Praspel

Les travaux présentés dans ce mémoire portent sur la validation de programmes PHP à travers un nouveau langage de spécification, accompagné de ses outils. Ces travaux s’articulent selon trois axes : langage de spécification, génération automatique de données de test et génération automatique de tests unitaires.La première contribution est Praspel, un nouveau langage de spécification pour PHP, basé sur la programmation par contrat. Praspel spécifie les données avec des domaines réalistes, qui sont des nouvelles structures permettant de valider etgénérer des données. À partir d’un contrat écrit en Praspel, nous pouvons faire du Contract-based Testing, c’est à dire exploiter les contrats pour générer automatiquement des tests unitaires. La deuxième contribution concerne la génération de données de test. Pour les booléens, les entiers et les réels, une génération aléatoire uniforme est employée. Pour les tableaux, un solveur de contraintes a été implémenté et utilisé. Pour les chaînes de caractères, un langage de description de grammaires avec un compilateur de compilateurs LL(⋆) et plusieurs algorithmes de génération de données sont employés. Enfin, la génération d’objets est traitée.La troisième contribution définit des critères de couverture sur les contrats.Ces derniers fournissent des objectifs de test. Toutes ces contributions ont été implémentées et expérimentées dans des outils distribués à la communauté PHP.The works presented in this memoir are about the validation of PHPprograms through a new specification language, along with its tools. These works follow three axes: specification language, automatic test data generation and automatic unit test generation. The first contribution is Praspel, a new specification language for PHP, based on the Design by Contract. Praspel specifies data with realistic domains, which are new structures allowing to validate and generate data. Based on a contract, we are able to perform Contract-based Testing, i.e.using contracts to automatically generate unit tests. The second contribution isabout test data generation. For booleans, integers and floating point numbers, auniform random generation is used. For arrays, a dedicated constraint solver has been implemented and used. For strings, a grammar description language along with an LL(⋆) compiler compiler and several algorithms for data generation are used. Finally, the object generation is supported. The third contribution defines contract coverage criteria. These latters provide test objectives. All these contributions are implemented and experimented into tools distributed to the PHP community

Grammar-Based Testing using Realistic Domains in PHP

International audienceThis paper presents an integration of grammar-based testing in a framework for contract-based testing in PHP. It relies on the notion of gtypes, that make it possible to assign domains to data, by means of contract assertions written inside the source code of a PHP application. Then a test generation tool uses the contracts to generate relevant test data for unit testing. Finally a runtime assertion checker validates the assertions inside the contracts (among others membership of data to gtypes) to establish the conformance verdict. We introduce here the possibility to generate and validate complex textual data specified by a grammar written in a dedicated grammar description language. This approach is tool-supported and experimented on the validation of web applications

Contract-based testing for PHP with Praspel

International audienceWe summarize several contributions related to the PHP Realistic Annotation and SPEcification Language (Praspel). This language extends PHP programs with annotations for the formal specification of the behavior of their functions and for the declaration of types for their data. These contracts are used to automate test generation, by deriving test cases and test data, and test execution, by checking assertions at run-time in order to establish the test verdict. Our approach to contract-based testing for PHP is fully implemented into a PHP framework currently in use by several web companies

Praspel: A Specification Language for Contract-Based Testing in PHP

International audienceWe introduce in this paper a new specification language named Praspel, for PHP Realistic Annotation and SPEcification Language. This language is based on the Design-by-Contract paradigm. Praspel clauses annotate methods of a PHP class in order to both specify their contracts, using pre- and postconditions, and assign realistic domains to the method parameters. A realistic domains describes a set of concrete, and hopefully relevant, values that can be assigned to the data of a program (class attributes and method parameters). Praspel is implemented into a unit test generator for PHP that offers a random test data generator, which computes test data, coupled with a runtime assertion checker, which decides whether a test passes or fails by checking the satisfaction of the contracts at run-time